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Hausdorff dimension of the set 
of nonergodic directions 

By Yitwah Cheung 
(with an Appendix by M. Boshernitzan) 

Abstract 

It is known that nonergodic directions in a rational billiard form a subset 
of the unit circle with Hausdorff dimension at most 1/2. Explicit examples 
realizing the dimension 1/2 are constructed using Diophantine numbers and 
continued fractions. A lower estimate on the number of primitive lattice points 
in certain subsets of the plane is used in the construction. 

1. Introduction 

Consider the billiard in a polygon Q. A fundamental result [KMS] implies 
that a typical trajectory with typical initial direction will be equidistributed 
provided the angles of Q are rational multiples of ir. More precisely, there is 
a flat surface X associated to the polygon such that each direction 8 £ S 1 
determines an area-preserving flow on X; the assertion is that the set NE(Q) 
of parameters 9 for which the associated flow is not ergodic has measure zero. 
The statement holds more generally for the class of rational billiards in which 
the (abstract) polygon is assumed to have the property that the subgroup of 
0{2) generated by the linear parts of the reflections in the sides is finite. For 
a recent survey of rational billiards, see [MT]. 

Let Q\, A E (0, 1), be the polygon described informally as a 2-by-l rectan- 
gle with an interior wall extending orthogonally from the midpoint of a longer 
side so that its distance from the opposite side is exactly A (see Figure 1). We 
are interested in the Hausdorff dimension of the set NE(Qa)- Recall that A is 
Diophantine if the inequality 

A— - <r i- 
q \q\ e 



has (at most) finitely many integer solutions for some exponent e > 0. 
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Figure 1. The billiard in Q\. 

Theorem 1. If X is Diophantine, then H.dimNE(Q A ) = 1/2. 

In fact, Masur has shown that for any rational billiard the set of nonergodic 
directions has Hausdorff dimension at most 1/2 [Ma]. This upperbound is 
sharp, as Theorem 1 shows. It should be pointed out that the theorems in 
[KMS] and [Ma] are stated for holomorphic quadratic differentials on compact 
Riemann surfaces. The flat structure on the surface associated to a rational 
billiard is a special case, namely the square of a holomorphic 1-form. 

The ergodic theory of the billiards Q\ was first studied by Veech [VI] 
in the context of 2,2 skew products of irrational rotations. Veech proved the 
slope of the initial direction 9 has bounded partial quotients if and only if the 
corresponding flow is (uniquely) ergodic for all A. On the other hand, if 9 has 
unbounded partial quotients, then there exists an uncountable set K{9) of A for 
which the flow is not ergodic. In this way, Veech showed that minimality does 
not imply (unique) ergodicity for these Z2 skew products. (The first examples 
of minimal but uniquely ergodic systems had been constructed by Furstenberg 
in [Fu].) Our approach is dual to that of Veech in the sense that we fix A and 
study the set of paramaters 9 € NE(Qa)- 

The billiards Q\ were first introduced by Masur and Smillie to give a 
geometric representation of the Z2 skew products studied by Veech. It follows 
from [VI] that NE(Q^) is countable if A is rational. A proof of the converse can 
be found in the survey article [MT, Thm. 3.2]. Boshernitzan has given a short 
argument showing H.dimNE((5A) = for a residual (hence, uncountable) set 
of A. (His argument is presented in the appendix to this paper.) Theorem 1 
implies any such A is a Liouville number. As is well-known, the set of Liouville 
numbers has measure zero (in fact, Hausdorff dimension zero). We remark that 
by Roth's theorem every algebraic integer satisfies the hypothesis of Theorem 1. 

Some generalizations of Theorem 1 are mentioned in Section 2. For the 
class of Veech billiards (see [V2]) the set of nonergodic directions is countable. 
It would be interesting to know if there are (number-theoretic) conditions on 
a general rational billiard Q which imply that the Hausdorff dimension of 
NE(Q) = 1/2. 

Theorem 1 can be reduced to a purely number-theoretic statement. 
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Lemma 1.1 (Summable cross products condition). Suppose (wj) is a 
sequence of vectors of the form (A + mj,rij), where mj,nj £ 2Z and rij ^ 0, 
and assume that the Euclidean lengths \wA are increasing. The condition 

(1) ^ \ W 3 X W 3 + l\ < °°> 

implies that 6j = Wj/\wj\ converges to some 9 6 NE(Q t x ) as j — > oo. (Here, 
Q\ is the billiard table obtained by reflecting Q\ in a line of slope — 1.) 

Theorem 2. Let K(X) be the set of nonergodic directions that can be 
obtain using Lemma 1.1. If X is Diophantine, then H.dimK(A) = 1/2. 

Proof of Theorem 1. Theorem 2 implies H.dimNE((5A) = H.dimNE(Q^) 
^ 1/2. Together with Masur's upperbound, this gives Theorem 1. □ 

Density of primitive lattice points. The main obstacle in our approach to 
finding lowerbounds on Hausdorff dimension is the absence of primitive lattice 
points in certain regions of the plane. More precisely, let E = E(a, R, Q) 
denote the parallelogram (Figure 2) 

E := Ux, y) £M 2 :\ya-x\^l/Q, R^y < 2i?} 

and define 

, , v x *{(p,q) g E : gcd(p, q) = 1} 

dens(E) := • 



2R 



R 







2/Q 




y 

y a 



Figure 2. The parallelogram E(a,R, Q). 
The proof of Theorem 2 relies on the following fact: 

Theorem 3. Let Spec(a) be the sequence of heights formed by the 
convergents of a. There exist constants Aq and pq > such that whenever 
area(E) ^ Aq 

Spec(a) n [Q, R}^$ =>- dens(E) ^ p . 
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Remark. It can be shown dens(E) = if a does not have any convergent 
whose height is between Q/4 and 8R. Thus, area(S) 3> 1 alone cannot imply 
the existence of a primitive lattice point in S. For example, the implication 



is easy to verify and remains valid even if | • | is replaced by the distance to the 
nearest integer (because arithmetic density is preserved under (J ")). 

Outline. Theorem 2 is proved by showing that K(A) contains a Cantor set 
whose Hausdorff dimension may be chosen close to 1 /2 when A is Diophantine. 
The construction of this Cantor set is based on Lemma 1.1 and is presented in 
Section 2. The proof of Theorem 2 is completed in Section 3 if we assume the 
statement of Theorem 3, whose proof is deferred to Section 4. 

Acknowledgments. This research was partially supported by the National 
Science Foundation and the Clay Mathematics Institute. The author would 
also like to thank his thesis advisor Howard Masur for his excellent guidance. 



We begin with the proof of Lemma 1.1, which is the recipe for the con- 
struction of a Cantor set E(\) C K(X). We then show that the Hausdorff 
dimension of E(X) can be chosen arbitrarily close to 1/2 if the arithmetic den- 
sity of the parallelograms T,(a,R, Q) can be bounded uniformly away from 
zero. 

2.1. Partition determined by a slit. The flat surface associated to Q\ is 
shown in Figure 3. It will be slightly more convenient to work with the reflected 
table Q\. Let X\ be the flat surface associated to Q l x . The proof of Lemma 1.1 
is based on the following observation: 

X\ is a branched double cover of the square torus T = R 2 /Z 2 . 

More specifically, let wq C T denote the projection of the interval [0, A] con- 
tained in the x-axis. X\ may be realized (up to a scale factor of 2) by gluing 
two copies of the slit torus T \w$ along their boundaries so that the upper 
edge of the slit in one copy is attached to the lower edge of the slit in the other, 
and similarly for the remaining edges. The induced map it : X\ — > T is the 
branched double cover obtained by making a cut along the slit wq. 

Lemma 2.1 (Slit directions are nonergodic). A vector of the form 
(A + m, n) with m, n € 2Z and determines a nonergodic direction in Q\. 




dens(S) = 



2. Cantor set of nonergodic directions 
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Figure 3. Unfolded billiard trajectory. 

Proof. A vector of the given form determines a slit w in T that is homolo- 
gous to wq (mod 2). (We assume A is irrational, for the statement of the lemma 
is easily seen to hold otherwise.) If it' : X' — > T is the branched double cover 
obtained by making a cut along w, then there is a biholomorphic isomorphism 
h : X\ — > X' such that tt = ir' o h. It follows that tt~ 1 (w) partitions X\ into 
a pair of slit tori with equal area, and that this partition is invariant under 
the flow in the direction of the slit. Hence, the vector (after normalization) 
determines a nonergodic direction in Q\. □ 

Proof of Lemma 1.1. It is easy to see from (1) that the directions 9j form 
a Cauchy sequence. The corresponding partitions of X\ also converge in a 
measure-theoretic sense: the symmetric difference of consecutive partitions is 
a union of parallelograms whose total area is bounded by the corresponding 
term in (1); summability implies the existence of a limit partition. Invariance 
of the limit partition under the flow in the direction of 9 will follow by showing 
that hj, the component of Wj perpendicular to 9, tends to zero as j — > oo ([MS, 
Th. 2.1]). To see this, observe that the area of the right triangle formed by 
Wj and 9 is roughly hj times the Euclidean length of Wj; it is bounded by the 
tail in (1) and therefore tends to zero. (We have implicitly assumed that A is 
irrational. For rational A the lemma still holds because a nonzero term in (1) 
must be at least the reciprocal of the height.) □ 

Remark. A vector of the form (A + m, n) with m, n € gZ and n ^ 
determines a partition of the branched ^-cyclic cover of T into g slit tori of equal 
area. From this, it is not hard to show that the conclusion of Theorem 1 holds 
in genus g ^ 2. Gutkin has pointed out other higher genus examples obtained 
by considering branched double covers along multiple parallel slits. Further 
examples are possible by observing that the proof of Theorem 2 depends only 
on a Diophantine condition on the vector wq = (A,0). (See §3.) 
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2.2. Definition of E(X). Our goal is to find sequences that satisfy condi- 
tion (1) and intuitively, the more we find, the larger the dimension. However, 
in order to facilitate the computation of Hausdorff dimension, we shall restrict 
our attention to sequences whose Euclidean lengths grow at some fixed rate. 

We shall realize E(X) as a decreasing intersection of compact sets Ej, 
each of which is a disjoint union of closed intervals. Let V denote the set 
of vectors that satisfy the hypothesis of Lemma 2.1. Henceforth, by a slit 
we mean a vector w £ V whose length is given by L := |n| and slope by 
a := (A + m)/n. Note that the following version of the cross product formula 
holds: \w x w'\ = LL'A, where A is the distance between the slopes. Fix a 
parameter 5 > 0. 

Definition 2.2 (Children of a slit). Let w be a slit of length L and slope a. 
A slit w 1 is said to be a child of w if 

(i) w' = w + 2{p, q) for some relatively prime integers p and q 

(ii) \qa-p\ sC 1/LlogL and q € [L 1+s ,2L 1+S }. 

Lemma 2.3 (Chains have nonergodic limit). The direction of Wj con- 
verges to a point in K(X) as j — > oo provided Wj + \ is a child of Wj for every j. 

Proof. The inequality in (ii) (equivalent to \w x w'\ < 1/logL) implies 
that the directions of the slits are close to one another. Hence, their Euclidean 
lengths are increasing since the length of a child is approximately L 1+s . The 
sum in (1) is dominated by a geometric series of ratio 1/(1 + 5). □ 

Choose a slit wq and call it the slit of level 0. The slits of level j + 1 are 
defined to be children of slits of level j. Let V := UVj where V- denotes the 
collection of slits that belong to level j. Associate to each w € V the smallest 
closed interval containing all the limits obtainable by applying Lemma 2.3 to 
a sequence beginning with w. Define E(X) := DEj where Ej is the union of 
the intervals associated to slits in V-. It is easily seen that the diameters of 
intervals in Ej tend to zero as j — ► oo. Hence, every point of E arises as the 
limit obtained by an application of Lemma 2.3. Therefore, E(X) C K(X). 

2.3. Computation of Hausdorff dimension. We first give a heuristic calcu- 
lation which shows that the Hausdorff dimension of K(X) is at most 1/2. (This 
fact is not used in the proof of Theorem 1.) We then show rigorously that the 
Hausdorff dimension of E(X) is at least 1/2 under a critical assumption: each 
slit in V' has enough children. 

Recall the construction of the Cantor middle-third set. At each stage of 
the induction, intervals of length A are replaced with m = 2 equally spaced 
subintervals of common length A'. In this case, the Hausdorff dimension is 
exactly log2/log3, or logm/log(l/e) where e := A'/A = 1/3. 
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For if (A) it is enough to consider sequences for which every term in (1) 
is bounded above. Associated to each slit of length L is an interval of length 
A = 1/L 2 . The number of slits of length approximately V is at most m = 
L'/L. Their intervals have approximate length A' = 1/(L') 2 . Therefore, 

log m 1 



H.dimif(A) < 



log(A/A') 2 



To get a lowerbound on the Hausdorff dimension of E(A) we need to 
show there are lots of children and wide gaps between them. The number of 
children is exactly 2L S / log L times the arithmetic density of the parallelogram 
S(a, R, Q) where R = L 1+s and Q = L log L. 

Lemma 2.4 (Slopes of children are far apart). The slopes of any two chil- 
dren of a slit with length L are separated by a distance of at least 0(1/L 2+2S ) . 

Proof. Let w be a slit of length L. A child w' has the form w' = w + 2v 
for some v = (p,q). If w" = w + 2v' is another child with v' = (p',q'), then 
v' v. Since both pairs are relatively prime, \p/q — p'/q'\ ^ 1/qq' ^ 1/4L 2+25 . 
The lemma follows by observing that the slope of w' satisfies 



/ P 
a 



\w' x v\ \w X v\ L\qa — pi 1 



L'q ~(L + 2q)q" 2q 2 ^ 2L 2+2S log L ' 



Proposition 2.5 (Enough children implies dimension 1/2). Suppose 
there exists c\ > such that every slit in V' has at least c\L 6 / log L children 
in V' , where L denotes the length of the slit. Then H.dim K(X) = 1/2. 

Proof. The length of a slit in V- is roughly Lj = L^ + ^ , where Lq denotes 
the length of the initial slit wq. The number of children is at least nij = 
ciLj/ log Lj and their slopes are at least Ej = 1/AL 2+2S apart. It follows by 
well-known estimates for computing Hausdorff dimension (we use [Fa, Ex. 4.6]) 
that 

H.dim£(A) > liminf ^o---^-i) = liminf E^log^ = ^ 

j^oo — log(mjEj) j^oo (2 + 6) log Lj 2 + 5 

Together with the upperbound on K(\), this proves the lemma. □ 

Remark. Theorem 3 allows us to determine when a slit has enough chil- 
dren. It should by pointed out that Diophantine A does not imply every slit 
will have enough children. We shall show that Proposition 2.5 holds if V is 
replaced by a suitable subset. (By the remark following Theorem 3 one can 
easily show there are slits that do not have any children and whose directions 
form a dense set.) 
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3. Diophantine condition 

Let wq be the initial slit in the definition of E(X). The hypothesis that A 
is Diophantine implies there are constants eo > and cq > such that 

\\wn x v\ \ = min \wq x v — n\ > , ° for all v E Z 2 , v ^ 0. 
11 11 nez 1 1 \v\ e o ' r 

Fix a real number N so that eo < NS. We assume the length of wo is at least 
some predetermined value Lq = Lo(X,S,N,eo,co). 

Definition 3.1 (Normal slits). A slit of length L and slope a is said to be 
normal if for every real number n,l^n<iV + l, 

Spec(a) n [e n5 L log L,L 1+nS ] + 0. 

Let V" be the subset of V' formed by normal slits of length ^ Lq. 

Proposition 3.2 (Normal slits have enough children). There exists c\ > 
such that every slit in V" has at least C\L 5 / log L children in V" . 

To complete the proof of Theorem 2 we also need 

Lemma 3.3 (Normal slits exist). Arbitrarily long normal slits exist. 

Proof of Theorem 2 assuming Lemma 3.3 and Proposition 3.2. We may 
choose the initial slit wo to lie in V", which is nonempty by the lemma. The 
calculation in the proof of Proposition 2.5 applies to a subset of E(X) to give the 
same conclusion; in other words, the proposition implies H. dim If (A) = 1/2. 

□ 

We recall two classical results from the theory of continued fractions. The 
k th convergent /qk of a real number a is a (reduced) fraction such that 



Pk 

a 

Qk 



(2) : ^ 

QkiQk+l + Qk) Qk QkQk+l 

and satisfies the recurrence relation qu+\ = a^+iqu + qk-i (similarly for p^), 
where afc is the k th partial quotient. A partial converse is that if p and q > 
are integers satisfying 

1 



p 

a 

q 



^ 2q 2 



(3) 

then p/q is a convergent of a, although it need not be reduced. 
3.1. Existence of normal slits. 

Definition 3.4. A slit of length L and slope a is said to be n-good if 
Spec(a) n [e n5 L log L,L 1+S ] + 0. 



HAUSDORFF DIMENSION OF NONERGODIC DIRECTIONS 



669 



Lemma 3.5. A sufficiently long N-good slit is normal. 

Proof. An (iV+l)-good slit is normal by definition, so it suffices to consider 
the case of an iV-good slit that is not (N + l)-good. Suppose w is such a slit, 
with length L and slope a. Let qk be the largest height in Spec(a) n [l,L 1+<5 ] 
so that q k = e niS L\ogL for some ri\ between N and N + 1. Set v := (Pk,Qk)- 
By the RHS of (2), the Diophantine condition, \v\ G 0(L log L) and eo < NS 
we get 

Qk+i < \qka-Pk\~ 1 < (l/c )L\vr ^ L 1+m 
provided L > Lq. Since N ^ n±, this shows w is normal. □ 

Proof of Lemma 3.3. By the previous lemma, it is enough to prove the 
existence of arbitrarily long iV-good slits. We show that a sufficiently long slit 
that is not iV-good has a nearby slit that is iV-good. 

Hence, let wbe a slit of length L and slope a and assume it is not iV-good. 
Let q k be the largest height in Spec(a) n [1,L 1+S ]. Since q k+1 > L l + 5 (h ere 
we use the irrationality of A to guarantee the existence of the next convergent) 
the RHS of (2) implies A := (L\q k a - pk])' 1 > L s . With V :=L + 2mq k , it is 
not hard to see that there exists a positive integer m satisfying 

e NS log(L') + 1/2 < A < (L 1 ) 5 . 

Indeed, if m is smallest for the RHS, then the LHS holds when L > Lq. 

Let w' = w + 2mv where v = (pk,Qk)- We show w' is iV-good. Let a 1 
be its slope. Using \w' x v\ = \w x v\ and the cross product formula, we find 
\a' —pk/Qk\ = 1/L'qkA ^ 1 / which by (3) implies qk € Spec(a'). Using the 
above inequalities on A in parallel with those in (2) we obtain 

q k+ i ^ L'A s$ (L') 1U and 

q k+1 > L'(A — qk/ L') > e NS L' log L' 

which show that w' is iV-good. □ 

3.2. Normal slits have enough normal children. Assume w is a normal slit 
of length L ^ Lq and slope a. Let qk be the largest in Spec(a) n [1, L 1+s ] and 
define n\ > 1 uniquely by q k = e niS Llog L. 

Lemma 3.6 (Enough children). Since w has at least 0(L S / log L) (n — 1)- 
good children where n := min(ni,./V + 1), if w' is a child with length V and 
slope a', then w' = w + 2(pk>,qk') and qy+i € [V log L', (L') 1+6 ] for some 
qk' € Spec(a'). 

Lemma 3.7 (Most children are normal). The number of children con- 
structed in the previous lemma that are not normal is at most 0(L S ~ S logL). 
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Proof of Proposition 3.2 assuming the above lemmas. A slit w G V" has 
enough normal children. These are all longer than Lq and therefore lie in V" . 

□ 

Proof of Lemma 3.6. Applying Theorem 3 with Q = e nS LlogL and 
R = L 1+<5 gives 0(L S / log L) children of the form w' = w + 2(p,q) where 
gcd(p,q) = l,q€ [L 1+S ,2L 1+S ] and \qa - p^ 1 ^ e nS L\ogL. 

Observe that L' = L + 2q so that \a' - p/q\ = L\qa - p\/L'q l/2q 2 . 
Since gcd(p, q) = 1, (3) implies q = qk> G Spec(a') for some index k'. 

It remains to bound the next height qy+i G Spec(a'). Using the LHS of 
(2) together with the lower bound on \qa — we have 

q k , +1 > L'/L\qa - p\ - q w > L'(e nS log L - 1/2) > e^ 5 L' log L' 

since L ^ Lo. Using the RHS of (2) and L ^ Lo again 

g fc /+i < L'/Lq\a-p/q\ < L'L" 5 < (L') 1+(5 . 

(In the second step we used the fact that \a — p/q\ ^ l/L 2+2<5 which, by 
Lemma 2.4, holds for all the children with a finite number of exceptions.) □ 



Proof of Lemma 3.7. Let V be the collection of slits formed by the children 
constructed in Lemma 3.6 that are not normal. We show that V has at most 
0(L 5 ~ S logL) elements. Observe that if n\ > N + 1, all children constructed 
are A-good, hence normal, by Lemma 3.5. In this case, V is empty and we 
have nothing to prove. Therefore, we may assume ri\ < N + 1. 

The next lemma will allow us to count the number of elements in V. 

Lemma 3.7. Let w' be a slit in V of length L' and slope a' . Then (i) 
the largest q v G Spec(a') n [1, {L') 1+s } lies in [V log L', e NS L' log L'] and (ii) it 
satisfies the inequality below for at most finitely many possible values of a. 

(4) \L(q v a-p v ) ± 2a\ < where n 2 ■= max(l, m - 1). 

Proof. By definition (ft'+i = (L') 1+n 5 for some n' ^ 1. Since u>' is (ni — 1)- 
good, (#/ = e n 5 L'\ogV for some n" > ni — 1 > 0. By Lemma 3.5, u/ is not 
A-good so that n" < A; this proves (i). 

Since n\ < A + 1, the fact that w' is not normal implies n! ^ n" ^ 712- 
Let G Spec(a') be as in Lemma 3.6 and recall qy+i G [V logL', (L') 1+<5 ]. 
By definition of qy, qy ^ Qfc'+i- The recurrence relations satisfied by conver- 
gents imply that qy = aq^+i + bqw for some integers a > and b ^ 0. By (i), 
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Write w' = w+2v and note that \w' x v\ = \wxv\. Since the cross product 
of consecutive convergents is ±1 (thought of as vectors), 

L'\q v a -pi>\ = \L(qi>a - p v ) ± 2a\. 

The RHS of (2), V = L + 2q > L 1+s , and n' > n 2 imply 

r/i / l L ' 1 1 



yt / +1 " (L 1 +< 5 )™'' 5 " £n 2 5+n 2 a 2 
and (ii) follows. □ 

Lemma 3.7 allows us to write V as a finite union of subsets V± a . Let Q± a 
denote the corresponding set of heights qy associated to the slits in V± a . The 
next two lemmas complete the proof of Lemma 3.7. □ 

Lemma 3.8. V± a and Q± a have the same number of elements. 

Proof. We need to show that the map V± a — > Q± a sending w' to qy is 
injective. Let w" be different from w' with corresponding image qy. Note that 
since \a! — py/qy \ ^ 1 / (L') 1+s qy is small compared to the distance between the 
slopes of w' and w" (Lemma 2.4), the rationals py /qy and py /qy are distinct. 
Their heights differ because the interval containing them is smaller than 1/qy. 

□ 

Lemma 3.9. Each Q± a is a union of at most O(logL) subsets, each 
having at most 0(L s ~ s2 ) elements. 

Proof. Let qy, qy £ Q± a and set q := \qy — qy\. We claim that if q < 
then q = dq k for some positive integer d £ 0(L 5 ~ 5 ). This implies the lemma 
since we either have q > L 1+s or q L 1+s so that the elements of Q± a fall 
into O(logL) clusters, each having 0(L 6 ~ &2 ) elements. 

Hence, assume q ^ L 1+s and set p = \pi — py\. Let p/q be the reduced 
form of p/q so that q = dq where d = gcd(p,q). From (4) and the triangle 
inequality 

2 

\ qQl ~ P \ L l+n 2 5+n 2 52 ' 

Since q ^ q ^ L 1+s , \a — p/q\ ^ l/2q 2 . By (3) p/q is a convergent of a; since 
gcd(p, (?) = 1, q = q^i G Spec(a) for some index fc'. By hypothesis, g < L 1+s , 
so we must have k' ^ fc. In fact, we must have k' = k because k' < k implies 
\qa— p\ = d\qk>a — pk>\ ^ l/(<7fc + Qk-i) ^ 1/2% which contradicts the previous 
inequality. Using the LHS of (2), we now have 

d = \W-P\ < 2{q k+ i + g fc ) 



|9fc«-Pfc| ^ L 1+n2<5 + n2<52 
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which is 0(L^ n3 ~ n2 ^~ n ' 2 ^ '') where 713 is defined by qu+i = L 1+n:iS . Since w is 
normal, 7X3 < n±. This together with (4) implies 723 — ri2 < n\ — n>2 ^ 1 and 
ri2 > 1; thus proving the claim. □ 



4. Counting rationals in intervals 

Primitive lattice points in the parallelogram £ correspond to rationals in 



the interval / : = 



a RQ ' a + RQ 



. Set 

A/ := {(x,y) € M 2 : x/y G /, P < y ^ 2P} . 
Theorem 4. //Spec(a) n [Q,P] / 0, dens(A/) ^ 1/24 and R/Q ^ 16. 

Proof 0/ Theorem 3. We use ylo = 16 and po = 1/32. Observe that £ 
contains A// where P is concentric with / and half as wide. Moreover, A// 
occupies three quarters of its area. Replacing Q with 2Q in Theorem 4 and 
assuming Spec(a) PI [2Q, R] 7^ 0, we conclude dens(S) ^ (3/4) dens(A//) ^ po- 

If Spec(a) n [2Q, P] = 0, let q k be the largest height in Spec(a) n [Q, R\. 
By the RHS of (2) it is easy to show the reduced fraction 

- = where a = 1,2,3,... 

Q aq k + g fe _i 

(known as an intermediate fraction of a when a < afc+i) satisfies 



(5) 



a 

q 



qk + |%+i - q\ 



q / q k qk+i 



These correspond to R/qu primitive lattice points in S, and since qk ^ 2Q, it 
follows easily that dens(S) ^ 1/4 ^ po- □ 

Lemma 4.1. // J /ias rational end-points of height at most R, dens(Aj) 
^ 1/6. 

Proof. We shall first prove the lemma under the additional hypotheses: 

(i) the height of any rational in int( J) is greater than R, and 

(ii) \pq' — p'q\ = 1, where p/q and p' jq' are the endpoints of J. 

By (ii), arithmetic density is preserved by the linear map 7 which sends the 
standard basis to lattice points corresponding to the endpoints of J. Note that 

7 _1 (Aj) C A := {(x,y) £ R 2 : x/a + y/a' ^ 2, x,y > o} 

where a = R/q and a' = R/q' ■ Let n(A) denote the number of primitive lattice 
points int(A). By (i), it is enough to show n(A) ^ aa' /4. 
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Without loss of generality, assume a' > a > 1. There are two cases. If 
a' < 2 then aa' ^ 4, and since (1, 1) € A we have n(A) ^ 1 ^ aa'/A. On the 
other hand, if a' > 2, then since 7(1, 1) G Aj, we have 1/a + l/a' ^ 1 so that 



n(A) > 



pairs of the 


form (1. 


n), 


we 


find 






'2 


1 






> aa' ^ 






-i) 






.a 


" a2 





aa' 



This completes the proof assuming the additional hypotheses. 

Note that every interval is a disjoint union of intervals satisfying (i). 
Hence, the lemma follows if we show (i) = (ii). Indeed, let d = \pq' — p'q\. 
There is a linear map in GL2Z that takes (p, q) to (0, 1) and (p', q') to (d, d!) 
for some integer d',0 ^ d! < d. If df > then (1,1) is contained in the 
triangle determined by the origin, (1,0) and (d, d!) and corresponds to a ra- 
tional of height at most R in int(J). Therefore, (i) implies d! = 0, and since 
gcd(d, d!) = 1, this in turn implies that d = 1, giving (ii). □ 

The height of a rational strictly between p/q and p'/q' is at least q + q': 



1 



qq 



p p" 
q q" 



+ 



p" p' 



1 1 



gg" g'g" 



This will be used several times in the next proof. 

Proof of Theorem 4. Let a' and a" denote the left and right endpoints 
of I, respectively. Let g& be the largest height in Spec(a) n [Q,R]- By (2), 
Pk/lk €= I and without loss of generality, we assume Pk/qk ^ ol. Let qi be the 
largest height in Spec(ct") n [1, R]. Since Pk/qk cannot lie strictly between pi/qi 
and pi+i/qi+i-, it must be the case that Pk/qk ^ Pi/qi- In fact, strict inequality 
must hold because qi = qk ^ Q and \a' — Pk/qk\ ^ 1/QlQl+i < ^/RQ give a 
contradiction. 

Let J := [pk/(lk-,Pi/(li\- We claim its length is at least 1/2RQ. In fact, 
Pl/qi lies within 1/2RQ of a" if g; ^ 2Q. On the other hand, qi ^ 2Q implies 
\J\ > 1/qkQl > 1/2RQ. In either case, \J\ ^ 1/2RQ. 

We may assume a" ^ Pi/qi, for otherwise J C I and dens(A/) ^ 1/24. 

There are three cases. First, if g/ + i + 3g/ > 2R there can be at most two 
rationals with height at most 2R that lie strictly between pi/qi and pi + i/qi + \: 

Pl+i Pi+i + Pi Pi+i + 2pi m 
Ql+i Qi+i + Qi Ql+i + 2 Qi Ql ' 

Let n(Aj) denote the number of primitive lattice points in A/. By Lemma 4.1, 
. . . area(Aj) „ / 1 2Q\ , A , 

and since i2/Q ^ 16, dens(A/) ^ 1/24. 
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Next, suppose qi + \ + 3qi < 2R and qi > 2Q. Note that an intermediate 
fraction of a" with height between q\ and qi + \ lies to the left of a". Let 
:= \pk/ 1k,p/ q\ where p/q is the intermediate fraction with the largest height 
not exceeding R. By definition, q > R — q%. From (5) we have 



P 

a 

q 



, i ' <ii + qi+i-q \ 1 < / 2fl-2 gt -<A 1 < 1 



V R-qi J 2RQ IRQ 

so that \J'\ > 1/2.RQ and dens(A/) > 1/24. 

Finally, assume that + 3^ < 2R and g/ < 2Q. Again, we consider 
the intermediate fractions of a". Observe that their heights are at most 2R, 
since qi + \ < 2R. They form a sequence that increase towards a" from the left. 
Given a consecutive pair with heights less than R we can always find a rational 
strictly in between them with height in [R, 2R\. It follows that the number of 
rationals in / with height in [R,2R] is at least (qi+i — q)/qi, where q is the 
height of the first intermediate fraction that falls into /. According to (5), an 
intermediate fraction lies in / as soon as its height is greater than 



R , , = (qi + qi+\)RQ 

2q lQl+1 + RQ • 



Hence, q — qi < R' and 



m oi 2qiqi+i + RQ 

^ 2R 2 -RQ 2 ( 5Q\ 

> 9RQ 1 = 27 l 1 - -R) ^ Kl) - 

Since R/Q > 16, we get dens(A/) > 1/24. □ 
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Appendix 

By Michael Boshernitzan 

With notation as in the introduction, for A <G [0, 1), set 

(6) h{\) d = H.dimNE(Q A ). 

Recall (see Introduction) that h(X) < 1/2 for all A [Ma], and, by Theorem 1, 
that h(X) = 1/2 for all Diophantine A. We also have h(X) = for rational A 
(then the set Q\ is in fact countable [VI]). The main result in this section is 
given by the following theorem. 
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Theorem 5. The set of A G [0, 1) for which H.dim NE(Q A ) = form a 
residual subset of [0, 1). In particular, there are irrational A G [0, 1) such that 
h(\) = 0. 

Recall that a subset A C X of a topological space X is called residual (or 
topologically large) if it contains a dense G^-subset of X. A subset Y C X is 
called a G^-set (in X) if 1" is a countable intersection of open subsets of X. 
Its complement X \ Y is called an F a -set. 

We remark that no irrational number A satisfying h(X) = is known even 
though the set is topologically large (in particular, uncountable). Any such A 
must be Liouville. (Note that the set of Liouville numbers forms a residual set 
of Lebesgue measure 0.) 

A.l. Z,2 skew products of irrational rotations 

Let X = Sq U S\ = S 1 x {0, 1} be the union of two unit circles S\ = 
S 1 x {n}, n G {0, 1}, and consider the two-parameter family of transformations 

p a>X -X^X, ael, XeK= [0,1), 

defined as follows. For x = (s,n) G X, 

(7) Pa,x(x) = p a ,x(s,n) = (s®a,n'), n' = 

where x = (s, n) G X, s, s © a G S 1 = R/Z = [0, 1), and © stands for the 
group operation in S 1 . 

The dynamical systems (X, p a ,\) have been studied by Veech [VI] as par- 
ticular Z2 skew product extensions of irrational a-rotations. Indeed, p a ^\ may 
be interpreted as the first return map to a disjoint union of two circles embed- 
ded in the surface associated to Q\. In particular, properties of billiards on 
Q\ reduce to the study of dynamical systems (X, p a ^\). One verifies that the 
ergodicity of the billiard system Q\ in direction 9 is equivalent to the ergodicity 
of the map p a ^\ with a = tan(#) (the slope in direction 9). Denote 

NE(A) = {(a, A) G R x [0, 1) | (X, p Q;A ) is not ergodic} , 

and, for A G [0, 1), 

(8) NE(AT A ) = {q G R I (a, A) G NE(A)} 

= {a G R I (X, p a ,x) is not ergodic} . 
The sets NE(X\), NE(Q A ) have the same Hausdorff dimensions because 
NE(A A ) = NE(X A ) + 1 = tan(NE(Q A )) = {tan(0) | 9 G NE(Q A )} . 
Thus we have (see (6)) 

(9) h(X) = H.dim NE(Q A ) = H.dim NE(A A ). 



if < s < A, 
n, if A < s < 1, 
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A. 2. The topological lemma 

The following lemma is central in the proof of Theorem 5. 

Lemma A.l. Let L be a G$-subset of a o-compact metric space K . Let 
P be a Polish space (a space with a complete metric topology). Let H be an 
F a -subset of the cartesian product W = K x P. For every p £ P, denote 

(10) K{p) = {keK \ (k,p) € H} C K 
and 

(11) Po = {p e P | K(p) C L}. 

Lf P Q is dense in P, then P Q is a residual subset (i.e., contains a dense 
Gs-subset) of P. In particular, P Q is uncountable if P is. 

Proof. Since the family of residual subsets of P is closed under coun- 
table intersections, we assume (as we may without loss of generality) that K 
is compact and H is closed in W = K x P. 

Since P is separable and P Q is dense in P, there is a countable subset 
P c C P which is dense in P. We assume that the points of P c are arranged in 
one sequence {pi} so that every point p € P c is repeated infinitely many times, 
i.e. pi = p, for infinitely many i > 1. 

Let 7rp: W — > P, ttk'- W —> K be canonical projections. Denote 

(12) Ki = K(p i )=TT K (TTp\ Pi )nH), 

(13) K'i = K i x{p i } = 7T P 1 ( Pi )riH) cLx{ Pi }. 
Since L is a G^-subset of K, L has a representation 

L = p| Ui, K DU 1 DU 2 DU 3 ... 

i>l 

where {Ui} forms (without loss of generality) a nonincreasing sequence of open 
subsets of K. 

Fix any integer i < 1. The set Hi = H \ -K^(Ui) C W is closed, and so 
is the set TTp(Hi) C P (the projection np: W — > P is a closed map since K is 
compact). One observes that pi £ irp(Hi) (since Ki C L C Ui), and hence 

Kp\pi) nH = Ki = K t x { Pl } C {Ui x { Pi }) n H C ttk 1 n H. 

Therefore the set V = rifc>i(Ui>A; ^) i s a dense G^-subset of P (it contains 
a dense subset P c ). 
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To complete the proof, we have to verify that V C P . If v G V, then 
v G V{, for an infinite set of i. For all those i, 

v i irp(Hi) = 7Tp{Hi n ir K \K \ [/;)) 

which implies K(v) C C/j. It follows that K(v) C L = Di>i ^i) an d therefore 
t> G P D . This completes the proof of Lemma A.l. 

A. 3. The completion of the proof 

Let TC be a separable Hilbert space. Denote by C(TL) the Banach algebra 
of bounded linear operators on H, and by U(TC) the subset of unitary operators 
on TC. We recall that convergence Tj — > T in the strong (or weak) operator 
topology means convergence Tj/ — ► Tf, for all f £ TC, in the strong (or weak, 
respectively) topology of the Hilbert space TC. The strong and weak operator 
topologies on CiTC) are different, but they coincide when restricted to the set 
U(TC) C C{TC) (see e.g. Halmos [Ha, pp. 61-80]). 

Now we view X = S 1 x {0, 1} as a probability measure space (X, B, fj) with 
fi a multiple of the Lebesgue measure, dn = ds/2. Take H = L 2 (X,B,fi), and 
denote by Q(X) the family of invertible measure-preserving transformations 
T : X -> X. Then Q{X) is naturally imbedded into U{H): for T € G(X) and 
f €H = L 2 (X,B,n) define T(/(x)) = /(T(x)) G W. The subspace topology 
on Q{X) induced by a strong (equivalently, weak) operator topology on U{TL) 
is called weak topology on G(X). The following result is well known (see [Ha, 
p. 80]). 

Lemma A. 2. The set 

S(X) = {T £ Q(X) | T is ergodic} C G(X) 

is a dense G$- subset of Q{X) (in the weak topology of Q(X)). 

Now let K = S 1 and P = [0, 1), and define W = K x P = S 1 x [0, 1), just 
as in Lemma A.l. The map 4> : W — > Q(X) defined by the formula (see (7)) 

4>(w) = Pw = pk, P , for w = (k,p) G W = K x P 

is easily verified to be continuous. In view of Lemma A. 2, the set (f)^ 1 (£(X)) 
is a G^-subset of W, and thus the complement 

H d = W \ _1 (5(X)) = {w = (k,p) G W | (f>(w) = p Kp is not ergodic} 

is an Fo-subset of W. 

Denote by Q(P), Q(AT) the sets of rationals in P and K = R/Z = [0, 1), 
respectively. Fix an arbitrary G^-subset L in K such that 



(14) 



Q(K) CLCK, 



H.dim L = 0. 
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To satisfy the conditions of Lemma A.l, it remains to verify that P is 
dense in P. For every A e P, we have K(X) = NE(A" A ) (see (8), (10)), and 
therefore 

P = {A <E P | NE(X A ) C L}. 

For A G Q(P), Veech [VI] proved that NE(X A ) = Q(if). Since L D Q(K) 
by the choice of L, the inclusion Q(-P) C P D holds. Thus P Q is indeed dense 
in P, and, by Lemma A.l, P Q is a residual subset of P = [0, 1). 

For every A <G P D , we have NE(A" A ) C L; thus H.dim NE(A" A ) = in view 
of (14). This completes the proof of Theorem 5. 

Rice University, Houston, TX 
E-mail address: michacl@math.rice.edu 
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